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Abstract 

Network-centered  Command  and  Control  (C2)  has  great  potential  to  increase  military 
effectiveness,  in  some  measure  due  to  enhanced  information  sharing  and  dissemination 
techniques.  However,  for  these  technologies  to  be  maximally  effective,  C2  organizations  need  to 
have  the  flexibility  to  tailor  their  organizational  structures  in  response  to  changing  mission 
conditions. 

In  the  experiment  reported  here,  a  model-based  approach  to  supporting  organizational  adaptation 
was  assessed.  The  purpose  of  this  experiment  was  to  explore  ways  in  which  obstacles  to 
adaptation  could  be  overcome.  Teams  of  Naval  Officers  participated  in  three  simulations  of  a  joint 
forces  mission  on  the  Distributed  Dynamic  Decision-making  (DDD)  simulator  (Serfaty  & 
Kleinman,  1985;  Kleinman  &  Serfaty,  1989).  The  match  between  organizational  structure  and 
mission  task  requirements  was  manipulated  within-participants,  resulting  in  differences  in 
coordination  requirements.  Between  the  second  and  third  simulated  missions,  participant  teams 
were  given  the  opportunity  to  select  an  organizational  structure  from  a  list  of  model-based, 
predefined  organizational  designs,  to  better  accommodate  the  changing  mission  requirements.  To 
support  organizational  change,  model-based  prospective  information  was  provided  to  the  teams. 
This  support  led  to  the  adoption  of  better  matched  congruent  organizations  in  each  of  the 
participant  teams.  Several  measurement  techniques  were  designed  to  evaluate  both  the  degree  of 
adaptation  and  its  effect  on  mission  performance. 
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Introduction 


The  emerging  network-centered  military  promises  an  unprecedented  degree  of  operational 
flexibility,  in  part  due  to  improved  information  sharing  and  dissemination.  Initiatives  such  as 
FORCEnet  require  that  information  from  warfighters  at  all  echelons  of  command,  sensors  attached 
to  unmanned  automated  vehicles,  and  automated  analyses  be  integrated  and  delivered  to  the 
appropriate  decision  makers  in  a  manner  that  significantly  improves  operational  tempo,  situational 
awareness,  and  mission  effectiveness.  Yet,  to  take  maximum  advantage  of  these  emerging 
technologies,  military  organizations  need  to  have  the  flexibility  to  quickly  adjust  their 
organizational  structures  (e.g.,  team  member  roles  and  responsibilities,  information  flow  patterns, 
allocation  of  resource  control)  in  response  to  changing  conditions  so  as  to  respond  to  emerging 
threats  as  quickly  and  effectively  as  possible.  Recent  exercises  (Diedrich  et  al.,  2003;  Entin  et  ah, 

2003)  have  demonstrated  that  mission  performance  is  higher  when  Command  and  Control  (C2) 
organizational  structures  are  aligned  -  or  congruent  -  with  mission  tasks  to  be  performed. 

Conversely,  when  organizations  are  incongruent  with  mission  tasks,  mission  performance  is 
degraded.  Congruence  is  defined  as  the  extent  to  which  resource  allocation  and  information  flow 
is  optimized  to  the  mission  at  hand.  Because  mission  effectiveness  is  of  such  high  criticality,  it  is 
imperative  that  organizations  recognize  the  degree  of  congruence  between  their  structure  and  the 
battlefield  requirements,  and  exercise  the  ability  to  modify  their  structures  if  appropriate.  This  has 
led  to  the  experiment  reported  in  this  report,  which  addresses  ways  to  support  organizational 
adaptation  and  the  development  of  model-based  measures  of  organizational  congruence  to 
facilitate  adaptation. 

Entin  &  Serfaty  (1999)  found  that  high-performing  organizations  demonstrate  improved 
coordination  when  they  recognize  high  degrees  of  incongruence.  Behaviors  and  communication 
patterns  change  strategically  in  response  to  changing  environmental  conditions,  improving  mission 
performance.  However,  teams  are  less  likely  to  alter  their  organizational  structure  to  increase 
organizational/environmental  congruence  (Hollenbeck  et  al.,  1999).  Here,  organizational  structure 
refers  to  the  allocation  of  assets  and  reporting  structures.  Several  reasons  have  been  noted  for  their 
reluctance: 

•  Lack  of  Authorization:  Military  organizations  do  not  have  a  history  of  rapid,  on-line 
structural  adaptation.  Without  this  precedence  for  self-induced  change,  organizations  may 
feel  that  they  lack  the  authority  to  make  alterations  to  their  structures. 

•  Lack  of  Training:  The  structure  of  organizations  cannot  be  designed  arbitrarily;  many 
parameters  need  to  be  carefully  considered  and  balanced.  Organizations  may  lack  the  training  to 
make  organizational  changes  effectively,  and  will  therefore  be  reluctant  to  change. 

•  Lack  of  Sensitivity:  When  organizations  sense  incongruence,  they  may  have  difficulty 
gauging  the  nature  or  severity  of  the  disparity.  As  a  consequence,  they  may  resist 
organizational  change  even  when  it  is  indicated. 

•  Lack  of  Familiarity:  Organizations  may  feel  uncomfortable  switching  from  established 
structures  to  those  that  are  less  well  known.  They  may  feel  that  imperfect  existing 
structures  are  more  desirable  than  structures  that  are  better  suited  but  unproven. 

As  a  consequence  of  these  obstacles  to  change,  organizations  may  choose  to  remain  in  inefficient 
structures,  neglecting  opportunities  for  improved  performance.  However,  the  potential  benefits  of 
alternative  organizations  may  warrant  their  consideration  even  in  the  face  of  these  obstacles. 
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Therefore,  it  is  essential  to  understand  how  an  organization  can  be  persuaded  to  adapt.  The  primary 
purpose  of  the  exploratory  study  reported  in  this  paper  is  to  address  this  question. 

A  secondary  purpose  of  this  study  was  to  identify  and  refine  measures  of  adaptation.  Entin  et  al 
(2003)  found  that  several  behaviors  consistently  occurred  with  high  organizational  incongruence, 
including  higher  self-reported  operator  workload,  larger  communications  volume,  and  certain 
performance  measures  intrinsic  to  the  simulation  environment  (i.e.,  tasks  processed  and  their 
accuracy).  These  factors  could  be  used  when  diagnosing  organizational  health  and  the  need  for 
structural  change  early  in  a  mission,  when  organizational  change  may  still  be  a  viable  course  of 
action.  It  should  be  recognized  that  measuring  the  indicators  that  underscore  the  need  for  structural 
adaptation  differs  from  measuring  the  occurrence  of  change  or  the  effects  of  change  in  subsequent 
performance.  Entin  et  al.  (2004)  investigated  inducements  toward  adaptation  where  the  mechanism 
that  allowed  structural  change  was  open  ended;  teams  of  Naval  officers  were  able  to  reallocate  control 
of  resources  in  response  to  changed  mission  conditions.  In  the  Entin  et  al.  study,  measuring  the 
occurrence  and  degree  of  change  was  as  straightforward  as  counting  the  number  and  types  of  resource 
allocation  exchanges;  the  fewer  the  number  of  exchanges,  the  smaller  the  degree  of  change.  However, 
while  this  may  be  an  indication  of  organizational  change,  organizational  adaptation  implies  that  the 
changes  made  improve  mission  performance.  As  a  consequence,  measures  must  be  developed  that  are 
sensitive  to  the  effects  of  adaptation,  even  in  simulations  where  overall  performance  is  obscured  by 
limited  training  opportunities.  In  the  current  study,  two  such  measures  were  employed,  one  based  on 
statistical  mission  performance  analysis,  the  other  based  on  modeled  team  performance. 

The  findings  presented  in  this  report  continue  a  long  line  of  research  (e.g.,  Entin  et  al.,  2003; 

Kleinman  et  al.,  2003;  Diedrich  et  al.,  2003;  Entin  et  al,  2004)  that  has  explored  aspects  of 
incongruence  between  organizations  and  their  mission  tasks.  In  several  experiments  (see  Diedrich  et 
al.,  2003),  variations  of  the  following  scenario  were  presented.  Two  organizational  designs  were 
utilized: 

•  A  divisional  organization  in  which  each  participant  had  complete  control  of  a  single 
multifunctional  platform,  and  was  therefore  able  to  independently  process  a  variety  of 
tasks  (e.g.,  air  interdiction,  surface  warfare,  search  and  rescue,  air  superiority)  in  a  bounded 
geographic  area. 

•  A  functional  organization  in  which  each  participant  had  complete  control  of  a  single 
mission  essential  function  (e.g.,  strike,  search  and  rescue,  special  operations)  throughout 
the  battlespace,  independent  of  the  platform  on  which  those  assets  resided. 

Two  mission  scenarios  were  created,  based  on  model-based  organizational  design  principles 
(Kleinman,  et  al,  2003),  which  were  designed  to  be  incongruent  to  one  of  the  organizational 
structures  while  providing  a  good  fit  (i.e.,  congruent)  to  the  other.  This  was  accomplished  by 
including  tasks  that  required  the  coordination  of  multiple  assets  that  were  controlled  by  either  a 
single  individual  (predicted  to  be  congruent,  regardless  of  organizational  structure)  or  multiple 
individuals  (predicted  to  be  incongruent,  regardless  of  organizational  structure).  Using  these 
organizations,  Diedrich  et  al  (2003)  found  that  performance  was  significantly  worse  in  the 
incongruent  conditions  as  compared  to  the  congruent  conditions,  based  on  a  variety  of  mission 
performance  and  behavioral  measures.  These  results  underscored  the  need  for  organizational 
adaptation  to  improve  mission  performance,  and  for  future  experimentation,  established  an 
appropriate  stimulus  for  change  (empirically  validated,  model-based  degradations  in  performance). 
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However,  encouraging  adaptation  is  a  difficult  task,  given  the  multiple  obstacles  to  organizational 
change.  Entin  et  al  (2004),  using  the  scenarios  and  organizations  described  above,  designed  a  study  to 
motivate  adaptation  by  providing  palpable  incentives  to  change.  Performance  deficiencies  were  noted 
and  presented  to  the  participants  with  the  express  purpose  of  motivating  adaptive  changes.  This 
provided  opportunities  for  facilitated  discussions  regarding  the  effects  of  organizational  change. 

Despite  these  actions,  structural  change  was  infrequent,  and  often  limited  to  a  few  modifications  of 
resource  allocation. 

Hence,  the  objectives  of  the  current  study  were  two-fold.  The  first  and  primary  purpose  was  to 
explore  the  mechanisms  that  can  help  decision  makers  identify  the  need  for  adaptation  and  enable 
them  to  make  adaptive  structural  changes.  In  particular,  the  methods  employed  were  designed  to 
overcome  the  obstacles  to  adaptation: 

•  Overcoming  a  perceived  lack  of  authorization:  Targeted  instruction  about  the  benefits  of 
organizational  change  was  presented  to  overcome  reluctance  to  adjustment  (i.e., 
overcoming  organizational  inertia). 

•  Overcoming  the  lack  of  training  in  organizational  design:  Rather  than  giving 
participants  the  ability  to  create  alternative  organizational  designs  from  scratch,  several 
fully  formed  organizational  designs  were  presented  to  participants,  accompanied  by  a 
rationale  for  their  creation. 

•  Overcoming  the  lack  of  sensitivity:  A  suite  of  feedback  measures,  based  on  proven 
performance  and  behavioral  indicators  of  congruence,  was  presented  to  participants  at 
critical  times.  The  display  design  and  timing  of  their  presentation  was  intended  to 
emphasize  the  criticality  of  organizational  adaptation. 

•  Overcoming  the  lack  of  familiarity:  Prospective  performance  measures,  based  on  proven 
modeling  techniques,  were  presented  to  accompany  unpracticed  organizational  design,  in  large 
part  to  lower  participant  anxiety  about  the  utility  of  novel  organizational  structure. 

A  secondary  purpose  of  this  experiment  was  to  identify  measures  of  structural  change  and  their  efficacy 
in  the  context  of  both  simulated  and  actual  mission  conditions.  Given  the  logistical  constraints  on 
participant  training,  these  measures  needed  to  be  robust  in  the  face  of  relatively  impoverished  mission 
performance.  Two  approaches  to  measurement  were  developed  toward  this  end: 

•  The  measurement  of  simulated  mission  performance  for  several  experimentally  inspired 
organizational  structures. 

•  The  measurement  of  participant  mission  performance  for  the  subset  of  tasks  that  required 
collaboration. 

These  methods  are  designed  to  facilitate  structural  change  by  supplying  the  knowledge  and  tools 
necessary  for  successful  adaptation,  and  to  measure  the  degree  of  adaptation  in  meaningful  ways. 

It  was  hoped  that  these  actions  would  foster  observable  adaptive  behaviors. 
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Method 


Participants 

Twenty-Five  officers  and  non-commissioned  officers  served  as  participants  for  this  exercise  (see 
Table  1).  Of  these,  18  were  Naval  Reservists  and  6  students  at  the  Naval  War  College  in  Newport, 
RI.  Participants  were  organized  into  four  teams  of  six  individuals. 

Table  1:  Participant  profile. 


Rank 

# 

Captain 

2 

Commander 

10 

Lieutenant  Commander 

7 

Lieutenant 

3 

Chief  Petty  Officer 

1 

Petty  Officer  2nd  Class 

1 

Simulation  Environment 

The  Distributed  Dynamic  Decision-making  (DDD)  environment  was  used  to  simulate  several 
mission  scenarios  for  use  in  this  study.  The  DDD  is  a  distributed  client/server  simulator  that  allows 
extensive  mission  customization  in  order  to  investigate  individual  and  team  performance  in  an 
operationally  rich,  experimentally  valid  environment.  In  general,  DDD  simulations  involve 
individual  and  team  decision-making  about  complex  situations  based  on  information  and  resources 
provided  by  both  the  simulation  and  other  team  members  (Serfaty  &  Kleinman,  1985;  Kleinman  & 
Serfaty,  1989).  The  simulation  enables  the  manipulation  of  variables  such  as  organizational 
structure  and  mission  scenario  tasking.  A  variety  of  performance  measures  can  be  recorded  within 
the  DDD  (i.e.,  tasks  processed,  latencies,  and  accuracies)  to  assess  performance. 

Procedure 

Each  team  was  assigned  to  one  of  two  organizational  structures  characterized  by  the  types  of  assets 
within  each  team  member’s  control  (see  Diedrich  et  al.,  2003).  In  the  Divisional  organizational 
structure  (D),  each  team  member  had  responsibility  of  functionally  diverse  assets  in  a  single, 
defined  geographic  area;  they  were  given  control  of  all  assets  within  that  area.  In  contrast, 
participants  in  teams  assigned  to  the  Functional  organizational  structure  (F)  were  given  control  of 
functionally  similar  assets  dispersed  throughout  the  mission  battle  space;  for  example,  they  might 
be  given  all  ISR  or  Strike  capabilities  available  within  the  simulated  environment.  In  this 
investigation,  participants  were  assigned  randomly  to  two  D  teams  and  two  F  teams. 

Participants  were  briefed  as  to  the  general  purpose  of  the  study.  Training  began  with  two  hours  of 
DDD  “buttonology”  in  which  participants  were  shown  how  to  use  the  DDD  user  interface,  the 
capabilities  of  the  assets  and  platforms  available  within  the  DDD  environment,  and  basic  strategies 
for  accomplishing  common  tasks.  This  was  followed  by  a  second,  two-hour  training  session  in 
which  participants  completed  short  mission  scenarios.  This  session  exposed  participants  to  the 
general  mission  expectations,  which  involved  following  a  mission  task  plan,  reacting  to 
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unexpected  time  critical  events,  balancing  defensive  and  offensive  roles,  and  coordinating  with 
other  team  members  for  task  completion.  The  training  scenarios  were  designed  to  instruct  without 
imposing  bias  for  any  particular  organizational  structure.  This  was  accomplished  by  creating 
hybrid  task  conditions  that  employed  aspects  of  both  divisional  and  functional  favoring  tasks.  The 
participants  performed  these  scenarios  in  both  the  D  and  F  organizations,  in  a  counterbalanced 
fashion. 

Following  the  second  training  session,  the  data-collection  phase  of  the  study  began.  The  two 
scenarios  used  were  developed  to  be  congruent  to  either  the  Divisional  organizational  structure  (D) 
or  to  the  Functional  organizational  structure  (F)  (Diedrich  et  al.,  2003).  A  mission  was  considered 
to  be  congruent  if  the  team’s  organizational  structure  (D  or  F)  was  matched  with  its  associated 
scenario  (i.e.,  a  D  team  in  a  d  scenario  or  an  F  team  in  an  f  scenario;  Table  2).  In  contrast,  a  session 
in  which  the  team  and  scenario  structures  are  mismatched  was  considered  incongruent  (D  with  f,  F 
with  d).  Congruence  was  achieved  by  defining  task  requirements  within  a  scenario  that  matched 
the  asset  capabilities  controlled  by  individuals  within  an  organizational  structure,  thus  reducing 
need  for  coordination  between  players  and  minimizing  communication  and  coordination  overhead. 

Table  2:  Congruence  of  Structure  and  Scenario  Requirements 


Organizational  Structure 

Divisional  (D) 

Functional  (F) 

Scenario 

Structure 

d 

Congruent 

Incongruent 

f 

Incongruent 

Congruent 

Each  mission  scenario  was  presented  in  operational  terms  that  were  familiar  to  the  participants.  They 
involved  a  Joint  Forces  mission  in  which  teams  were  required  to  use  a  variety  of  sea,  land,  and  air 
assets  to  destroy  or  capture  an  enemy  command  center,  two  naval  bases,  two  air  bases,  and  a  seaport. 
Concurrent  to  these  primary  mission  tasks,  the  teams  had  both  offensive  and  defensive  responsibilities 
that  were  time  critical  and  distracted  them  from  the  main  mission  objectives  (e.g.,  destroy  Exocet 
missiles  and  coastal  defense  launchers,  perform  search  and  rescue  operations,  engage  targets  of 
opportunity,  etc.).  For  additional  complexity,  the  area  of  operations  also  contained  neutral  parties  and 
peripherally  hostile  assets  which  were  either  to  be  ignored  or  to  be  of  low  priority.  Missions  lasted 
approximately  35  minutes  or  until  the  final  mission  task  was  completed,  whichever  came  first. 

Each  team  participated  in  three  data-collection  sessions  (see  Table  3).  In  the  first  data-collection 
session,  each  team  participated  in  a  scenario  congruent  with  their  organizational  structure.  In  the 
second  data-collection  session,  each  team  participated  in  a  scenario  which  was  incongruent  with 
their  organization.  To  justify  the  change  in  mission  task  requirements  associated  with 
incongruence,  teams  were  informed  that  their  adversary  was  changing  their  defenses  to  impede  the 
team’s  mission  progress.  However,  participants  were  not  given  the  opportunity  to  change  their 
structure  in  response;  they  had  to  participate  in  the  session  in  a  sub-optimal  incongruent  structure. 
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Table  3:  Experimental  Design 


Session  # 

Session 

Description 

Team  Structure 

i 

Congruent 

Team  structure  and  mission  task 
requirements  in  alignment 

Divisional 

or 

Functional 

2 

Incongruent  1 

Team  structure  and  mission  task 
requirements  in  discord 

Divisional 

or 

Functional 

Planning  Session:  Opportunity  to  Change  Structure  in  Response  to  Incongruence 

3 

“Incongruent”  2 

Congruence  between  Team 
Structure  and  Mission  Task 
Requirements  dependent  on 
Chosen  Team  Structure 

One  of  5  alternative 
organizations  (D,  D2, 

R2,  F2,  or  F) 

After  each  data-collection  session,  participants  were  given  feedback  on  several  aspects  of  their 
performance  via  a  computer  generated  “Congru-o-meter.”  The  Congru-o-meter  included  measures  of 
overall  performance,  performance  on  major  offensive  and  defensive  responsibilities,  participant 
workload,  and  gain  (a  composite  measure  of  task  accuracy  and  proportion  of  tasks  completed).  These 
measures  were  presented  in  comparison  to  high  performing  (congruent)  groups  described  in  Diedrich 
et  al  (2003).  The  criticality  of  organizational  change  was  underscored  in  the  presentation  of  these 
feedback  measures. 

Following  the  second  data-collection  session,  participants  were  given  the  opportunity  to  change  their 
organizational  structure.  Teams  were  given  explicit  guidance  on  the  merits  of  organizational  change 
and  its  expected  performance  gains.  During  a  45  minute  facilitated  planning  session,  five  fully- 
formed  alternative  organizational  structures  (see  Table  4)  were  presented  to  the  participants, 
purported  to  be  from  the  Joint  Task  Force  Commander  (CJTF).  After  making  their  organizational 
choice,  a  third  data  collection  session  commenced  in  which  the  mission  tasks  and  requirements  were 
identical  to  those  of  the  second  data-collection  session  (i.e.,  incongruent  with  the  team’s  original 
structure)  but  with  the  organizational  structure  chosen  during  the  facilitated  planning  session. 

Organizational  Structures 

In  a  previous  study  (Entin  et  al.,  2004),  teams  of  Naval  officers  were  asked  to  create  organizational 
structures  adaptively,  in  response  to  changing  degrees  of  congruence.  In  a  facilitated  planning 
session  lasting  approximately  one  hour,  these  teams  had  to  acknowledge  the  incongruence,  weigh 
the  appropriate  factors,  and  come  to  a  consensus  as  to  the  best  organizational  design.  In  the  face  of 
the  obstacles  described  in  the  introduction,  teams  are  reluctant  to  make  organizational  changes, 
even  when  presented  with  convincing  evidence  of  the  benefits  of  doing  so.  As  a  result  teams  made 
few  major  changes;  adaptation  was  modest. 

In  the  current  study,  further  efforts  were  made  to  encourage  adaptation.  Rather  than  requiring 
teams  to  create  organizational  structures  from  the  bottom  up,  fully  formed  alternative  organizations 
were  presented.  During  the  planning  session,  descriptions  of  alternative  organizations  were 
accompanied  by  several  supporting  documents,  including  a  description  of  the  roles  and 
responsibilities  of  each  team  member  within  that  structure  and  a  detailed  account  of  the  asset 
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allocation.  During  the  planning  session,  participants  had  to  discuss  the  relative  merit  of  each 
alternative,  and  then  arrive  at  a  consensus  regarding  their  preference. 


Table  4:  Organizational  Design  Choices 


Organizational 

Structure 

Description 

Divisional  (D) 

Each  participant  controls  a  single  platform  with  multiple 
functional  areas 

Divisional/Functional 
Hybrid  (D2) 

Four  participants  control  a  single  platform  each;  two 
players  control  functional  assets  across  the  theater  (ISR, 
BMD) 

Regional  (R2) 

Theater  is  divided  into  two  geographic  regions.  Groups 
of  three  participants  divide  the  assets  functionally 
(ISR/SAR/SuWC,  STRIKE/ AWC,  MINES/BMD) 
within  those  two  regions. 

Functional/Divisional 
Hybrid  (F2) 

Four  participants  control  functional  assets  (STRIKE, 
SOF/SAR,  BMD,  ISR)  across  the  theater;  two  players 
control  a  single  platform  each  with  multiple  functional 
areas. 

Functional  (F) 

Each  participant  controls  a  single  function  across  the 
theater 

These  alternative  organizations  were  created  by  holding  the  number  of  assets  within  the  game 
constant  (e.g.,  number  of  ships,  number  of  missiles)  while  changing  the  asset  control  allocation. 
The  original  D  and  F  structures  constituted  two  of  the  available  organizations.  Design  of  the 
remaining  three  organizations  was  constrained  by  several  factors.  First,  each  structure  needed  to 
have  an  operational  justification  -  it  needed  to  be  a  plausible  alternative  for  the  participants,  who 
had  extensive  Naval  expertise.  Second,  the  offered  organizations  had  to  represent  distinct 
alternatives  to  the  D  and  F  structures  discussed  above.  Participants  had  to  be  able  to  recognize  the 
advantages  and  disadvantages  of  each.  Finally,  the  new  organizational  structures  needed  to  be 
evaluated  with  regards  to  the  divisional  and  functional  scenarios,  and  the  degree  of  congruence 
determined.  Thus,  the  task  requirements  of  each  scenario  had  to  be  considered  when  designing  the 
organizational  structure. 

Three  new  organizational  structures  were  created  with  these  constraints  in  place.  They  were,  in 
part,  inspired  by  the  comments  made  by  operational  participants  in  a  previous  study  (Entin  et  al., 
2004).  The  first  structure  is  a  Divisional/Functional  Hybrid  (D2),  in  which  the  majority  of  team 
members  control  assets  in  a  manner  similar  to  the  Divisional  structure  (i.e.,  control  of  all  assets  on 
one  platform),  but  where  two  participants  are  given  more  extensive  control  of  a  single  asset  type 
(i.e.,  all  UAVs  or  anti-ballistic  missiles).  A  second  alternative  organization  is  a  variation  on  the 
Functional  organization.  In  the  Functional/Divisional  Hybrid  (F2)  organization,  four  team 
members  retained  essentially  Functional  roles;  they  controlled  a  single  asset  type  distributed 
among  the  platforms  simulated.  The  two  remaining  participants  played  a  Divisional  role,  each 
controlling  all  assets  on  a  single  platform.  The  third  structure,  the  Regional  (R2)  structure,  divided 
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participants  into  two  sub-groups  each  responsible  for  half  of  the  battlespace.  Within  each  of  the 
halves,  participants  had  defined  Functional  roles.  In  each  of  the  three  alternative  organizations,  the 
dispersion  of  resources  among  the  team  members  fell  in  between  the  Divisional  and  Functional 
organizations  (see  Figure  1). 

The  similarity  of  the  alternative  organizations  to  the  true  Functional  and  Divisional  organizations 
was  determined  by  measuring  the  degree  of  asset  control  dispersion  (Figure  1).  This  was 
determined  based  on  the  resource  capabilities  of  the  assets  controlled  by  each  team  member,  or 
decision-maker  (DM),  in  the  organization.  A  high  score  indicates  an  organization  in  which 
decision-makers  control  resources  of  distinct  functions  and  operational  capabilities.  This 
organization  corresponds  to  a  Functional  resource  allocation.  A  low  score  indicates  that  decision¬ 
makers  control  assets  with  similar  resource  capabilities.  This  indicates  an  organization  with 
Divisional  resource  distribution.  Formally,  this  metric  is  defined  as  follows. 

For  each  asset  m  in  the  organization  of  M  decision-makers  ( m  =  1  ,  where  L  is  the  number 

of  assets),  the  resource  capability  vector  is  defined  as  r'f ;  j  =  1  ,  where  K  is  the  number  of 

resource  types  (functions).  The  total  number  of  resources  for  a  type  j  is  therefore  equal  to 

m= 1 


The  asset  control  is  defined  via  DM-asset  assignment  variables  ai  m;i  =  1  =  1  ,  where 

ai  m  =  1  if  decision-maker  i  controls  asset  m,  and  ai  m  =  0  otherwise.  Then,  decision-maker  V s 
control  of  resource  type/function  j,  equal  to  the  amount  of  corresponding  resources  controlled  via 

L 

ownership  of  its  assets,  is  found  as  /?'  =  m  .  The  average  number  of  resource  of  type  j  at  any 

m  =  1 


decision-maker  is  -D-  =  — — .  This  defines  “averagely  capable”  decision-maker,  and  is  the  goal  of 
M  M 


Divisional  asset  distribution.  However,  the  feasibility  of  asset  assignment  might  prevent  such  an 

1  M  (  L  ft  V 

allocation.  The  error  function  is  defined  as  s , ,  = ,  — V  V  rJV  m  — -  ,  which  is  a  root-mean 

J  1  \/f  Z—t  Z—t  J  \/t 


square  error  for  distribution  of  resources  of  type  j  in  the  organization  and  identifies  the  dispersion 
of  resource  of  type  j  among  decision-makers.  This  value  is  0  for  divisional  resource  distribution 
(when  all  decision-makers  have  the  same  amount  of  resources  of  this  type,  thus  achieving 


maximum  overlap  in  resource  /),  and  equal  to  (M  - 1) 


for  functional  resource  distribution 


(when  only  a  single  decision-maker  controls  all  resources  of  type  j,  thus  achieving  no  resource 
overlap  in  resource  j). 
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As  the  result,  the  degree  of  asset  control  dispersion  is  defined  as  the  normalized  mean  square  error 


M 

2>,r 


function  s  = 


i= 1 


K  M  L 

m 

7=1  z=l  \vm=l 


7  M 


(M-i)jr 


7=1 


'V2 

vMy 


^  (  R 

M(M-l)Y}  — 

j= 1 


.  This  measure  is  equal  to  0  when  there  is  a 


vMy 


maximum  overlap  of  resource  capabilities  among  decision-makers  (Divisional  case),  and  equal  to 
1  when  there  is  no  overlap  in  resources  (Functional  case). 


Degree  of  Asset  Control  Dispersion 


Organizational  Structure 


Figure  1 :  Degree  of  Asset  Control  Dispersion. 

A  high  score  indicates  a  highly  Functional  organization  in  which  team  members  control  a  single 
asset  type  across  the  simulation;  a  low  score  indicates  more  consolidated  control. 


Applying  this  metric  to  the  original  and  hybrid  organizations,  the  Functional  organization 
displayed  the  most  dispersed  asset  control;  control  of  each  asset  type  was  given  solely  to  a  single 
individual.  Similarly,  the  Divisional  organization  displays  the  least  disparity  between  DMs, 
because  each  member  controls  a  limited  number  of  all  assets.  The  D2,  R2,  and  F2  organizational 
structures  demonstrated  a  degree  of  dispersion  or  spread  that  was  intermediate  between  the  D  and 
F  organizations.  Note  that  although  the  F2  organization  was  based  on  the  F  organization,  the 
degree  of  spread  was  mitigated  by  the  shared  responsibility  of  some  assets  between  all  DMs.  As  a 
consequence,  its  spread  was  lower  than  that  of  the  D2  and  R2  organizations. 

The  two  original  scenarios  (f  &  d)  were  designed  to  emphasize  disparities  in  congruence  between 
the  two  original  structures  (F  &  D).  Similarly,  the  congruence  of  the  three  new  structures  needed 
to  be  evaluated  with  regard  to  the  two  scenarios.  Two  aspects  of  congruence  were  modeled: 
coordination  and  overall  performance.  As  expected,  the  fully  Functional  and  fully  Divisional 
organizational  structures  were  modeled  to  exhibit  the  lowest  workload  (Figure  2)  and  highest 
performance  (Gain  Area;  refer  to  Figure  3)  in  the  functional  and  divisional  scenarios  respectively. 
The  other  three  organizations  were  modeled  to  lead  to  a  coordination  requirement  and 
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performance  in  between  these  two  endpoint  organizational  structures.  These  prospective,  model- 
based  data  were  shown  to  the  participants  as  they  made  decisions  regarding  potential  change. 


Figure  2:  Modeled  Coordination  Workload  Measure.  Lower  values  indicate  lower  overhead,  and 

are  therefore  lower  predicted  workload. 


Figure  3:  Modeled  performance  measure.  Higher  values  indicate  higher  expected  performance. 
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The  coordination  workload  (Figure  2)  was  modeled  to  predict  the  amount  of  communication 
among  decision-makers  required  to  synchronize  their  assets  to  execute  tasks.  This  synchronization 
is  needed  when  multiple  assets  owned  by  different  decision-makers  must  execute  the  same  task. 
The  normalized  coordination  workload  between  decision-makers  n  and  m  was  defined  as 

N 

D(m,n)  = - — - ,  where  N  is  the  number  of  tasks,  U  is  the  duration  of  task  i  and 

mission  duration 

1,  if  DM  m  is  assigned  to  task  i  1,  if  3  asset  j  such  that  amj  =  1  and  this  asset  executes  task  i 
0,  otherwise  [0,  otherwise 

can  be  seen  that  D(m,n )  is  equal  to  the  average  number  of  synchronized  task  executions  between 
decision-makers  n  and  m  per  unit  of  time.  The  coordination  workload  of  decision-maker  m  is 

M 

defined  as  E(m )  =  ^Z)(m,n) ,  and  the  coordination  workload  measure  of  an  organization  as  the 

n  =  \,n^m 

root-mean-square  of  coordination  workloads  of  DMs  in  this  organization:  0  = 

This  metric  accounts  for  both  the  mean  and  the  variance  of  coordination  workload  among 
decision-makers,  and,  consequently,  provides  a  measure  of  balance  of  coordination  among  DMs. 

In  the  presence  of  time-critical  tasks,  an  organization  may  trade-off  task  accuracy  for  timeliness. 
For  an  organization  that  is  incongruent  with  its  mission,  such  engagement  practices  may  result  in 
the  same  levels  of  task  timeliness  as  for  a  congruent  organization.  However,  this  timeliness  is 
achieved  at  the  cost  of  lower  accuracy.  Therefore,  a  measure  that  reflects  the  task  accuracy  and 
timeliness  tradeoff  can  be  combined  into  a  single  measure,  called  the  task  gain.  The  task  gain  of  a 
task  i  is  defined  as  the  accuracy  multiplied  by  its  value:  g;  =  at  ■  coi ,  where  af  is  task  accuracy  and 
coi  is  task  value.  Task  accuracy  is  based  on  satisfaction  of  task  resource  requirements  by  the  assets 

L 

assigned  to  execute  this  task.  When  all  resources  required  by  task  i  are  met,  that  is  •  r'"  >  rUj 

m= 1 


(where  yim 


1,  if  asset  m  is  assigned  to  task  i  A  , 
0,  othenvise  and  r< 


is  requirement  of  resource  type  j  for  task 


/),  then  the  accuracy  of  task  completion  is  equal  to  100%.  However,  in  realistic  applications  where 
the  resources  are  scarce,  an  organization  may  wish  to  reduce  the  task  execution  accuracy  in  order 
to  achieve  better  timeliness.  In  order  to  accommodate  timeliness-accuracy  trade-off,  a  model  of 
accuracy  has  been  defined  within  the  DDD  simulator  for  scoring  the  DMs  as  a  square  of  the 
average  of  ratios  of  the  resource  used  to  the  resource  required  over  all  resource  types: 


where  K  is  the  number  of  non-zero  resource  requirement  types 


of  task  i :  K  =  |  {  / :  rt  J  *  0}  | .  The  ratio  of  the  resource  used  to  the  resource  required  identifies  the 


percentage  of  satisfied  resource  for  the  corresponding  resource  type.  The  squaring  of  the  average 
of  these  ratios  for  required  resources  penalizes  significant  resource  allocation  mismatches. 
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Applying  these  measures  to  the  organizational  structures  used  in  this  study,  the  F  and  D  organizations 
were  determined  to  have  the  highest  congruence  with  the  scenarios  used.  The  F  organization  in  the  f 
scenario  and  the  D  organization  in  the  d  scenario  were  modeled  to  have  the  highest  gain  and  lowest 
coordination  workload  given  the  alternatives.  The  D2,  F2,  and  R2  were  by  and  large  intermediate 
between  these  endpoints,  showing  gain  and  workload  requirements  that  were  not  optimal  but  were 
preferable  to  the  fully  incongruent  organizations.  The  exception  to  this  was  the  fit  between  the  R2 
organization  and  the  f  scenario,  which  was  modeled  to  have  gain  that  was  comparable  to  the  congruent 
F  organization.  However,  it  was  also  expected  to  require  a  larger  degree  of  coordination,  which  could 
ultimately  lower  performance. 

Operational  justifications  were  created  for  the  scenarios,  based  on  their  relationship  to  the  F  or  D 
scenarios  already  familiar  to  the  participants.  The  relative  merits  of  each  organization  were  described 
in  terms  of  specific  roles  and  responsibilities  associated  with  each  team  member  in  each  organization. 
These  measures  indicate  that  the  new  organizational  structures  (D2,  F2,  &  R2)  have  levels  of 
congruence  intermediate  between  F  and  D  in  each  of  the  scenarios  used  in  this  study.  Therefore,  these 
intermediate  organizations  met  all  of  the  criteria  for  viable  alternatives. 

Results  &  Discussion 

While  the  DDD  is  capable  of  capturing  many  aspects  of  mission  performance  (e.g.,  accuracy,  task 
completion  latency,  mission  success,  subjective  workload),  the  focus  of  this  discussion  is  on 
structural  adaptation  to  change.  We  concentrated  on  two  major  components  to  the  assessment  of 
adaptation: 

1)  Did  adaptation  occur?  During  the  facilitated  planning  sessions,  teams  could  have 
chosen  to  remain  in  their  original  organizations.  The  range  of  alternative 
organizations  presented  varied  in  terms  of  congruence  to  the  anticipated  mission 
scenario.  It  was  therefore  critical  to  capture  the  choices  of  each  team,  and  note  the 
congruence  of  the  chosen  organizations. 

2)  Did  adaptation  improve  performance?  Change  is  ultimately  adaptive  only  if 
performance  improves  as  a  result  or  if  it  is  held  constant  in  the  face  of  increasing 
stress.  However,  given  logistical  constraints,  standard  performance  measures  (i.e., 
overall  mission  progress,  accuracy,  and  latency)  were  not  ideal.  Alternative 
measures  of  adaptation  were  explored  as  a  consequence. 

Noting  the  Occurrence  of  Adaptation 

During  the  facilitated  practice  session,  each  of  the  four  teams  chose  one  of  the  alternative 
organizations  (see  Table  5).  Both  of  the  teams  originally  assigned  to  the  Divisional  structure  chose  the 
Functional  organization.  This  change  demonstrated  maximum  adaptation  because  the  Functional 
structure  was  modeled  to  have  the  highest  congruence  with  the  scenario  they  were  to  engage. 
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Table  5:  Planning  Session 


Team 

Original 

Organizational 

Structure 

Chosen 

Organizational 

Structure 

Comments 

Team  1 

Functional 

D2 

Nearly  congruent 

Team  2 

Divisional 

Functional 

Fully  congruent 

Team  3 

Divisional 

Functional 

Fully  congruent 

Team  4 

Functional 

D2 

Nearly  congruent 

Similarly,  the  teams  originally  assigned  to  the  Functional  structure  chose  the  D2  organization.  The 
D2  structure  was  more  congruent  to  the  subsequent  scenario  than  the  original  Functional 
organization,  and  it  was  modeled  to  lead  to  more  efficient  coordination  and  higher  performance  than 
the  majority  of  the  alternative  organizations.  While  there  were  not  enough  teams  to  make  a  definitive 
conclusion,  it  is  possible  that  the  Functional  teams  were  reluctant  to  adopt  the  fully  congruent 
Divisional  organization  because  each  team  member  would  have  been  required  to  learn  a  larger 
variety  of  tasks.  In  the  Functional  organization,  each  team  member  has  only  one  or  two 
responsibilities,  and  thus  only  needs  to  learn  a  few  series  of  actions  to  interact  with  the  simulator.  In 
the  fully  Divisional  organization,  each  team  member  is  responsible  for  a  larger  variety  of  tasks  (~6- 
8).  The  task/organization  mapping  for  the  D2  organization  required  most  team  members  to  perform 
4-5  different  tasks,  while  two  team  members  were  responsible  for  a  single  task  each.  Participants 
may  have  been  compromising  between  the  modeled  gain  and  workload  information  presented  and 
the  anticipated  re-learning  required  for  the  transition  to  the  Divisional  organization. 

Measuring  the  Effects  of  Adaptation 

Each  team  made  organizational  changes  when  they  had  the  opportunity,  and  the  changes  made 
were  predicted  to  be  adaptive,  based  on  modeled  performance.  However,  organizational  change  is 
only  adaptive  if  it  leads  to  improved  performance.  Expert  performance  in  the  DDD  simulation 
requires  substantial  targeted  training  in  a  single  organizational  structure.  Because  the  purpose  of 
this  study  was  to  investigate  organizational  flexibility,  training  was  divided  between  both  D  and  F. 
This  reduced  bias  towards  particular  organizational  designs,  but  also  precluded  us  from  training 
participants  in  all  organizational  structures  to  the  degree  necessary  for  true  mastery,  leading  to 
wide  performance  difference  among  teams  and  generally  sub-optimal  simulation  performance. 
Therefore,  overall  simulation  performance  is  not  an  accurate  measure  of  adaptation. 

The  DDD  simulation  is  composed  of  a  moderately  high  number  of  events  (-130  per  session)  that 
team  members  must  engage,  either  individually  or  in  conjunction  with  other  team  members. 
Engagement  consists  of  placing  the  correct  asset(s)  within  a  proscribed  range  from  the  target, 
selecting  the  target,  and  prosecuting  the  target  via  a  series  of  menu  choices  -  all  while  avoiding 
enemy  attack.  Coordination  with  other  team  members  requires  communicating  the  intended 
actions,  and  timing  the  attacks  to  ensure  that  they  all  occur  within  a  thirty  second  window  of  time. 
Approximately  50%  of  events  required  a  single  team  member  to  engage  successfully  with  a  single 
asset,  while  the  remaining  50%  require  multiple  assets  to  engage  accurately. 

Recall  that  the  modeled  performance  of  the  organizational  choices  was  dependent  on  the 
coordination  requirements  dictated  by  control  of  resources.  Tasks  requiring  a  single  asset  are  not 


15 


associated  with  high  degrees  of  coordination  -  an  individual  team  member  can  engage  the  task 
easily.  However,  that  subset  of  tasks  requiring  multiple  assets  for  engagement  are  associated  with 
a  high  degrees  of  coordination.  In  a  highly  incongruent  environment,  these  tasks  require  multiple 
team  members  to  coordinate  resources  within  a  small  window  of  time,  increasing  overhead  and 
potentially  decreasing  performance  accuracy.  In  a  congruent  environment,  these  tasks  require 
single  individuals  to  use  multiple  assets  within  their  own  control.  The  predicted  result  is  higher 
accuracy. 

This  leads  to  predictions  about  performance  in  the  sub-set  of  tasks  that  require  coordination.  The 
scenarios  used  in  sessions  2  and  3  were  identical;  for  the  teams  that  began  in  the  Functional 
structure,  the  same  divisional  scenario  was  used,  while  teams  which  began  in  the  Divisional 
structure  participated  in  the  functional  scenario.  The  major  difference  between  session  2  and  3  was 
the  organizational  structure  of  the  teams.  Both  Divisional  teams  choose  new  Functional  structures, 
while  the  Functional  teams  chose  the  sub-optimal,  but  adaptive  D2  organization.  This  leads  to 
several  predictions  about  performance.  Regardless  of  overall  performance,  teams  should  engage  a 
larger  number  of  tasks  that  require  coordination  between  sessions  2  and  3.  The  accuracy  of  these 
engagements  should  improve  as  well.  These  improvements  should  be  greater  for  the  Divisional 
teams  (Teams  2  and  3)  because  their  choice  of  organization  was  more  congruent  to  the  scenario. 


Figure  4:  Percentage  of  Collaborative  Tasks  Attacked  in  Sessions  2  &  3 

Between  sessions  2  and  3,  the  percentage  of  tasks  that  required  collaboration  (that  were  engaged) 
increased  for  all  teams  (see  Figure  4).  This  includes  tasks  that  were  engaged  with  fewer  than  the 
required  assets  (i.e.,  the  task  required  both  special  forces  [SOF]  and  STRIKE  assets  [e.g.,  FI 8s 
fighters],  but  only  SOF  assets  were  used).  The  percentage  of  collaborative  tasks  engaged  increased 
on  average  by  12.75%  (7%  for  Functional  teams,  13.4%  for  Divisional  teams).  The  organizational 
choices  of  each  team  increased  the  structure/scenario  congruence,  which  decreased  the 
coordination  overhead.  This  allowed  teams  to  engage  more  targets.  This  effect  was  greater  for  the 
teams  originally  assigned  to  the  Divisional  organizations,  because  their  organizational  choice  (i.e., 
Functional)  had  a  higher  degree  of  congruence  with  the  scenario  than  did  the  teams  originally 
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assigned  to  the  Functional  organizations  (i.e.,  because  they  chose  the  sub-optimal  D2 
organization). 

A  smaller  proportion  of  tasks  requiring  multiple  assets  were  engaged  with  100%  accuracy  (see 
Figure  5).  A  clear  distinction  can  be  seen  between  the  Functional  and  Divisional  teams.  Overall, 
the  coordination  accuracy  of  the  Divisional  teams  increased  by  7.65%,  while  the  accuracy  of  the 
two  Functional  teams  remained  flat.  This  pattern  is  consistent  with  our  predictions;  with  higher 
congruence,  the  Divisional  teams  increased  their  accuracy.  The  Functional  teams  chose  a  structure 
that  was  less  congruent  with  the  scenario  (the  D2  structure  in  the  d  scenario).  As  in  Figure  3,  the 
modeled  gain  measures  -  an  indication  of  performance  -  suggested  that  D2  structure  would  lead  to 
only  modest  performance  gains.  The  baseline  performance  differences  between  groups  were  likely 
a  product  of  variable  computer  experience  among  participants. 


Figure  5:  Percentage  of  Collaborative  Tasks  Attacked  with  100%  accuracy  in  Sessions  2  &  3 

While  the  small  number  of  teams  and  the  high  degree  of  variability  precludes  meaningful  inferential 
statistics,  the  overall  trends  in  the  data  are  clear.  When  teams  choose  organizational  structures  that 
have  a  higher  degree  of  congruence  with  their  missions,  engagements  on  tasks  requiring  multiple 
assets  increases,  as  does  accuracy  on  those  tasks.  In  future  studies,  the  longer  term  benefits  of 
organizational  change  should  be  targeted,  so  temporary  decrements  in  performance  are  not  confused 
with  mismatches  between  organizations  and  scenarios. 

Conclusions 

The  transition  to  network-centered  warfighting  brings  with  it  the  possibility  of  flexibly  altering 
organizational  structures  and  resource  control,  creating  organizations  that  are  agile,  responsive, 
and  more  effective.  However,  before  this  promise  can  be  reached,  a  deeper  understanding  of  the 
implications  of  organizational  adaptation  must  be  attained.  The  current  study  is  part  of  a  series 
addressing  this  issue,  and  represents  a  substantial  step  forward.  Using  a  mix  of  experimentation 
and  model-based  simulations,  the  impact  of  organizational  change  on  performance  was  assessed. 

In  the  current  study,  indications  of  adaptive  structural  changes  were  found.  Four  teams  of  Naval 
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officers  chose  alternative  organizations  that  were  more  congruent  to  the  scenarios  when  given  the 
proper  tools,  experience,  and  rationale. 

In  a  similar  study  (Entin  et  al  2004),  similar  teams  were  reluctant  to  make  organizational  changes. 
Four  types  of  obstacles  were  identified  that  made  change  difficult,  all  of  which  were  addressed  in 
the  current  study.  These  barriers  to  change  were  addresses  in  the  following  manner: 

•  Overcoming  a  perceived  lack  of  authorization:  Specific  instructions  were  given 
regarding  the  criticality  of  organizational  change,  and  explicit  authorization 

•  Overcoming  the  lack  of  training  in  organizational  design:  Presenting  teams  with  model- 
based  alternative  organizations  that  were  already  balanced  allowed  participants  to 
concentrate  their  efforts  on  the  strategic  merits  of  each,  rather  than  details  about  low-level 
resource  allocation.  This  resulted  in  more  productive  discussions. 

•  Overcoming  the  lack  of  sensitivity:  The  use  of  the  congru-o-meter  allowed  participants  to 
easily  recognize  the  need  for  organizational  change,  and  contextualized  their  performance. 
Model-based  measures  of  structure/scenario  congruence  provided  a  baseline  that  was 
robust  to  local  variability  in  performance,  and  therefore  afforded  direct  comparison  of 
various  organizational  alternatives. 

•  Overcoming  the  lack  of  familiarity:  Barring  increased  opportunities  to  practice  in 
alternative  organizations,  model-based  prospective  information  about  novel  organizational 
design  allowed  participants  to  make  rapid  assessments  of  the  likely  efficacy  of  alternative 
organizations. 

Identifying  change  in  this  experimental  design  is  simple;  if  teams  chose  alternative  organizations, 
the  change  is  obvious.  However,  measuring  the  impact  of  that  change  is  more  difficult.  Adaptive 
changes  in  organizational  structures  should  lead  to  improved  team  performance.  However,  limited 
practice  in  the  simulator  environment,  inexperience  in  multiple  organizational  structures,  and 
variability  in  participant  computer  experience  lead  to  highly  variable  results.  As  a  result, 
alternative  measures  need  to  be  employed  to  evaluate  adaptation. 

The  modeled  performance  measures  used  to  evaluate  the  alternative  organizational  structures 
provide  an  efficient  way  to  evaluate  the  efficacy  of  alternative  organizations.  Entin  et  al  (2004) 
required  participants  to  create  new  organizational  designs  without  explicit  guidance  from  trained 
organizational  scientists.  Subsequent  DDD  performance  in  these  organizations  was  hampered  by 
limited  practice,  which  made  evaluation  of  the  utility  of  the  organizations  themselves  difficult. 
However,  calculating  modeled  performance  measures  is  not  subject  to  these  difficulties.  Model- 
based  measures  of  performance  and  workload  provide  an  alternative  evaluative  technique  when 
human  performance  is  affected  by  other  factors. 

In  the  current  design,  the  organizational  structures  were  presented  to  participants  fully  formed,  and 
the  modeled  performance  was  already  known.  While  performance  was  still  highly  variable, 
concentrating  analyses  on  the  subset  of  data  specifically  sensitive  to  coordination  proved  to  be  a 
valuable  analysis.  Although  performance  was  low,  the  change  in  performance  between  pre-  and 
post-adaptation  sessions  reflected  the  benefits  of  congruence.  In  future  studies,  these  measures  will 
be  refined  further,  allowing  analysis  of  the  numbers  of  tasks  requiring  intra-  and  inter-person 
coordination  while  investigating  the  shorter-term  costs  and  longer-term  benefits  of  organizational 
adaptation. 
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The  A2C2  Project 


The  General  Problem: 

->  How  would  you  design  a 
command  team  organization  for 
this  mission? 


How  would  you  derive  human  requirements  for  the  organization? 
How  would  you  evaluate  its  performance  for  this  mission? 
How  would  you  design  adaptability  into  this  organization? 
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The  A2C2  Project 


■  Objectives 

■  Develop  &  test  theory  of  adaptive  architectures 

■  Investigate  fit  between  organizational  structures  &  mission 
scenarios 

•  Incongruence  as  a  motivation  to  alter  organizational  structure 

■  Create  conditions  of  incongruence  &  observe  the  adaptation 
process 

•  Facilitators  &  inhibitors 


Theoretical 

Concept 


Experiment  & 
Data  Analysis 

Background 
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Hvmirt  - 


Congruent  teams  outperform  incongruent  teams 

•  Model-based  organizational  design  for  congruence  (Diedrich  et  al., 
2003) 


Organization 


Mission  vs.  Organization 


Mission 


•  Mission  effectiveness  would  be  enhanced  through 
organizational  adaptation. 

•  However,  teams  are  reluctant  to  change  their  structures  to 
achieve  congruence  (Entin  et  al.,  2004) 

•  Adaptations  observed  were  modest  and  variable 

•  Most  changes  were  small,  some  not  adaptive 

•  Participants  often  recognized  the  need  for  organizational  change, 
but  were  reluctant  to  do  so 
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Inducing  Change 


i  How  can  structural  adaptation  be  supported? 

■  Lack  of  Authorization:  Organizations  may  feel  that  they  lack  the 
authority  to  make  alterations  to  their  structures. 

•  Provide  Targeted  instruction 

■  Lack  of  Training:  Organizations  may  lack  the  training  to  make 
organizational  changes  effectively,  and  will  therefore  be  reluctant  to 
change. 

•  Provide  Fully  formed,  sound  organizational  designs 

■  Lack  of  Sensitivity:  Organizations  may  resist  organizational  change 
even  when  it  is  indicated. 

•  Provide  “Congru-o-meter”  to  signal  change 

■  Lack  of  Familiarity:  Organizations  may  feel  uncomfortable  switching 
from  established  structures  to  those  that  are  less  well  known. 

•  Model  based  prospective  performance  measures 


Divisional  (D) 


Design: 

Organizational  Structures 
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Functional  (F) 


1 

2 

3 

4 

5 

6 

Platform 

STRIKE 

ISR 

AWC 

SuWC/MINES 

SOF/SAR 

1 

CVN 

2F18S 

XXX 

1UAV 

2F18A,  E2C 

1FAB,  1MH53 

1HH60 

2 

8TLAM 

3ABM,4TTOM 

1UAV 

6SM2 

1FAB,  2HARP 

1HH60,1SOF 

3 

DDGB 

8TLAM 

3ABM,4TTOM 

1UAV 

6SM2 

1FAB,  2HARP 

1HH60,1SOF 

4 

CG 

8TLAM 

3ABM 

1UAV 

6SM2 

1  FAB,2HARP,1  MH53 

1HH60 

5 

2F18S 

XXX 

1UAV 

2F18A,E2C,4SM2 

1  FAB,2HARP,1  MH53 

1HH60 

6 

DDGC 

8TLAM 

3ABM,4TTOM 

1UAV 

6SM2 

1FAB,  2HARP 

1HH60,1SOF 

Asset  “ownership”  and  control  shape  team  structure: 
«  Multi-function  vs.  single-function  responsibilities 
■  Geographic  Area  of  Responsibility:  Local  vs.  Global 
Heterarchical,  not  Hierarchical,  organization 
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Design: 

Manipulating  Congruence 


■ 


■ 


Congruence  Manipulation 

*  Capitalizes  on  Roles  and 
Geography  (task  and 
asset  locations) 

Strategies  for  Manipulation 
of  Congruence 

»  Coordination 
Requirements 

■  Task  Phasing 

»  Limited  Assets 
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Example: 
Functional  (f)  Scenario 


Defend 
own  assets 


Obstacles 

to  SOF 
. *\ 

•  RGF 


2SOF+ 

2FAB 


3SOF 


2SOF+2FAB 


*  indicates  that  these  must  be  distinguished  from  neutral  (or  decoy)  counterparts 


=  aggregated  defend  task,  showing  possible  subtasks 
=  aggregated  encounters  task,  with  possible  subtasks 
=  mission  tasks  (that  must  be  done);  known  in  advance 
GEVA  may  spawn  as  a  result  of  performing  task 


TASK  RESOURCE  REQMTS 

SDG:  2_ASuW 
SPT,  SPH:  1  ASuW 
SGUN:  2  FAB 
SSAR:  2SAR 
SMIN:  2  MINES 

GEVA:  2SAR 

GCDL,  GSML:  1  STRK 

GSAM:  2  TLAM  (from  2  different  platforms) 

GSA3:  2  STRK  (1  F18S) 

GSA6:  2  TLAM  (from  2  different  platforms) 
GRGF :  3  STRK 

AAC,  APH,  ACDM,  AXOC:  1  AAW 
ACAP:  3AAW 
AMIS:  1  ABM 

-  other  unanticipated  tasks  via  HELP 
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Example: 
Divisional  (d)  Scenario 


Defend 
own  assets 


Obstacles 
to  SOF 

1SOF+2STRK 

r . *% 

|  • RGF  ! 

f  blovv\a 

p\bridgeT/ 

L . 

=  aggregated  defend  task,  showing  possible  subtasks 
□  =  aggregated  encounters  task,  with  possible  subtasks 
o  =  mission  tasks  (that  must  be  done);  known  in  advance 
=  GEVA  may  spawn  as  a  result  of  performing  task 


1SOF+2STRK 


1SOF+2STRK  +1FAB 


TASK  RESOURCE  REQMTS 

SDG:  1  ASuW_  +  7  A4I/1/ 

SPT,  SPH:  1  ASuW 
SHOS:  1  SAR  +  1  FAB 
SSAR:  1  SAR  +  1  FAB 
SMIN:  1  MINES  +  1  F18A 

GEVA:  1  SAR  +  1  F18A 
GCDL,  GSML:  1  STRK 
GSAM:  1_  TLAM  +  1_  SOF 
GSA3:  2  STRK  (1  F18S) 

GSA6:  2  TLAM  (from  2  different  platforms) 
GRGF\  2  STRK 

AAC,  APH,  ACDM,  AXOC:  1  AAW 
ACAP:  2AAW 
AMIS:  1  ABM 

-  other/unanticipated  tasks  via  HELP 


indicates  that  these  must  be  distinguished  from  neutral  (or  decoy)  counterparts 
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Consultants 


■  Study  run  at  the  Naval  War  College 

■  July  &  August,  2004 

■  Four  teams  of  highly  trained  consultants 

■  Three  Naval  Reserve  teams 

■  One  NWC  student  team 


Rank 

# 

Captain 

2 

Commander 

10 

Lieutenant  Commander 

7 

Lieutenant 

3 

Chief  Petty  Officer 

1 

Petty  Officer  2nd  Class 

1 
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Experimental  Design 


Session 

Description 

Team  Structure 

Congruent 

Team  Structure  and  Mission 
Task  Requirements  in 
alignment 

Divisional 

or 

Functional 

Incongruent  1 

Team  Structure  and  Mission 
Task  Requirements  in 
discord 

Divisional 

or 

Functional  i 

Planning  Session:  Opportunity  to  Change  Structure  in  Response  to 

Incongruence  j 

“Incongruent”  2 

Congruence  between  Team 
Structure  and  Mission  Task 
Requirements  dependent  on 
Chosen  Team  Structure 

Divisional 

D2-  Divisional/Functional  Hybrid 
Regional 

F2-  Functional/Divisional  Hybrid 
Functional 
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Organizational  Structures 


Divisional  -  Each  participant  controls  a  single  platform 

i  D2  -  Divisional/Functional  Hybrid  -  Four  participants 
control  of  a  single  platform  each;  two  players  control  functional 
assets  across  the  theatre 

i  Regional  -  Theatre  is  divided  into  two  geographic  regions. 
Groups  of  three  participants  divide  the  assets  functionally 
within  those  two  regions. 

■  F2  -  Functional/Divisional  Hybrid  -  Four  participants 
control  functional  assets  across  the  theatre;  two  players 
control  of  a  single  platform  each 

i  Functional  -  Each  participant  controls  a  single  function 
across  the  theatre 
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Prospective  Information  and  Congru-o-meter 


■  Performance  feedback  provided  the  second  and  third 
missions  to  encourage  adaptation. 

■  Several  Measures  Presented 

■  Performance  (Percent  Tasks  Completed) 

■  Perceived  Workload 

■  Gain 


Coordination  Workload  Measure 


■  Prospective  information 
given  before  Planning 
Session,  based  on 
models 

■  Gain 

■  Coordination  Workload 


~o 

<0 

o 


D2 


Organization 


I  ■  a 


F2 


R2 
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Results:  Adaptation 


Team 

Original 

Structure 

Chosen 

Structure 

Comments 

Team  1 

Functional 

D2 

Nearly  congruent 

Team  2 

Divisional 

Functional 

Fully  congruent 

Team  3 

Divisional 

Functional 

Fully  congruent 

Team  4 

Functional 

D2 

Nearly  congruent 
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Results:  Performance 


Percentage  of  Collaborative  Tasks  Attacked  in  Sessions  2  &  3 
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Results:  Performance 
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Session  2  (Pre-adaptation)  Session  3  (Post-Adaptation) 

Session 


Percentage  of  Collaborative  Tasks  Attacked 
with  100%  accuracy  in  Sessions  2  &  3 


Conclusions 
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i  Supporting  Adaptation  for  Mission  Effectiveness 

■  Teams  will  adapt  their  organizational  structures  if 
given  the  authority,  education,  incentive,  and 
information  to  do  so  effectively 

i  Library  of  Organizational  Designs 

■  Creation  of  several  model-based  organizational 
structures  allowed  rapid  adaptation 

Measuring  Adaptation 

■  The  extent  and  impact  of  adaptation  can  be  measured 
using  tailored  metrics 
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Thank  You 
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Extra  Slides 


Experiment  9  Pilot 
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Experiment  9  Pilot 


Teams  of  participants  at  the  Naval  Postgraduate  School 


Follow  directly  from  Experiment  8  results 

■  Observe  and  assess  adaptation  in  response  to  incongruence 

•  Will  an  organization  that  is  in  an  incongruent  situation  recognize  this  fact, 
and  adapt  its  structure  (e.g.,  who  owns  what,  who  does  what)  in  order  to 
become  more  “congruent”  with  its  environment? 

■  Evaluate  our  ability  to  induce,  guide,  support  and  measure  strategy 
and  structural  adaptation  via: 

■  Training,  procedures,  triggers,  feedback,  decision  aids,  ... 


i 


Affect  adaptation  during  facilitated  off-line  planning  sessions,  not 
during  on-line  dynamic  play 
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Pilot  9  Design  (1) 


Buttonology  and 
“hash”  training 


Phase  1  (3hrs) 
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H  is  a  hybrid  organization,  “midway”  between  D  and  F 
i  Exposes  players  to  elements  of  Functional  &  Divisional  structures 


First  “adaptation”  (F  =>  FI,  etc.)  to  external  SCUD  threat 
■  Requires  players  to  allocate  new  assets  (TTOM,  ABM)  and  new  roles 
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Pilot  9  Design  (2) 


Phase  2  (3hrs) 


“Homework”  given 
to  prepare 


Observational  and  self-reporting  instruments  were  designed  to 
collect  data  during  planning  sessions 

Feedback  and  aids  utilized  during  planning  sessions 
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Planning  Session  Protocol 

■  Planning/adaptation  occurs  prior  to  the  next  play 

■  Questions  posed  to  team  by  facilitator: 

•  How  are  we  doing? 

•  Should  we  adapt  in  some  way?  If  yes,  how? 

■  Discussions  were  recorded  for  analysis 

•  Asset  changes  recorded  for  immediate 
implementation 

■  Strategic  changes  and  rationale  recorded 


Aptima 

A  Hum  mii  -  t  iirtNi 


Hvmin  -  CericD^tf  £nfi>n&*ririf 


Congru-o-meter 


Provided  feedback  before  each  planning  session  based  on  team 
performance  to  encourage  adaptation 
■  Model-Based  Performance  data  was  available  within  minutes. 


Measures  displayed  were  suggested  by  previous  study  as  leading 
indicators  and/or  model  based 

■  Performance  (Percent  Tasks  Completed) 

■  Perceived  Workload 

■  Communications  Distribution 

•  Gain  (UCONN) 

*  Cognitive  Demand  (CMU) 
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Adaptation  Analysis 


>  To  assess  how  adaptive  changes  made  by  the 
teams  were,  we: 

»  Broke  asset  allocation  into  the  smallest  meaningful 
elements 

■  Measure  percent  overlap  between  team  asset  allocation 
for  each  mission  with  the  modeled  organization 


The  result  is:  overall  similarity  between  the  team¬ 
generated  allocations  and  the  modeled  class  of 
organizations 
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Divisional  Results 


Hypothesized  Divisional  — >  Functional 


Initial  SCUDS/PS1  PS2  PS3 

Session 


Team  C 


Initial  SCUDS/PS1  PS2  PS3 


Session 


Team  A  adapted  in 
anticipation  of 
incongruence 

Team  C  made  few, 
minor  changes. 

Team  E  changed  in 
a  maladaptive 
fashion. 


Team  A 


SCUDS/PS1  PS2 

Session 


Team  E 


Blue:  Percentage  of  Assets  that  are  Functional 
Red:  Percentage  of  Assets  that  are  Divisional 
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Team  B  made  adaptive  changes  in 
reaction  to  incongruence. 


Team  D  made  few,  minor  changes. 


Blue:  Percentage  of  Assets  that  are  Functional 
Red:  Percentage  of  Assets  that  are  Divisional 


Functional  Results 


Team  B 


Initial  SCUDS/ PS1  PS2 
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Manipulating  Congruence  (2) 


■ 


■ 


Congruence  Manipulation 

*  Capitalizes  on  Roles  and 
Geography  (task  and 
asset  locations) 

Strategies  for  Manipulation 
of  Congruence 

»  Coordination 
Requirements 

■  Task  Phasing 

»  Limited  Assets 
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Previous  Experiments 
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Diedrich  et  al.,  2003 


■  Realization  of  Model-based 
Experimentation 


•  Model-based  Organizations  (F,  D) 

•  Scenarios  (f,d) 

•  Successful  Manipulation  of  Congruence 


•  Congruent  out-performed  Incongruent 

•  Differences  in  communications  & 
workload  -  leading  indicators  of 


incongruence 

•  New  Model-Based  Measures 
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Entin  et  al.,  2004 


Demonstration  of  structural  adaptation  in  some  teams  based  on 
model-based  congruence  manipulation 
»  Open-ended  adaptation 

■  Adaptations  observed  were  modest  and  variable 

•  Most  changes  were  small,  some  not  adaptive 

■  Participants  often  recognized  the  need  for  organizational  change,  but 
were  reluctant  to  do  so 

■  Implementation  of  initial  version  of  congru-o-meter 

■  Model-based  measures  available  for  planning 
»  Near  real-time  information  available 

■  Observations  indicated  that  more  detailed  performance  feedback  would 
be  beneficial 


